Skip to content

generative_gap, and describe the lifted search as restricted to lifted generators - #31

Merged
Autoplectic merged 3 commits into
mainfrom
lifted-generator-wording
Oct 23, 2026
Merged

Autoplectic merged 3 commits into
mainfrom
lifted-generator-wording

Conversation

@Autoplectic

@Autoplectic Autoplectic commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #29.

generative_gap (missed #29's merge)

generative_gap(measure="exact" | "wyner"), also BidirectionalEpsilonMachine.generative_gap(), returns GenerativeGap(lower, upper, gap, model): the unconstrained common information of (S+, S-) against its value over lifted generator states. For the exact measure G[S+:S-] <= C_g <= H[G], so gap bounds C_g - G[S+:S-] from above. On the (2,∞) process: exact 1.406 → 1.5, Wyner 1.129 → 1.311; zero for first-order Markov processes. This commit was pushed to #29's branch after it was merged, so it is cherry-picked here unchanged.

Lifted generators are a proper subclass of generators

#29 described the constrained optimizers as searching "generator states". They search lifted generator states: the HMM built by applying p(g | s+, s-) at both ends of every bidirectional edge draws G_0 and G_1 independently given the edge, and a general generator may couple them.

Counterexample: for Z_t = Z_{t-2} xor Bernoulli(0.1) (two interleaved first-order chains), the product of the chains' minimal generators has state G_t = (U_t, U_{t+1}), sharing U_{t+1} with G_{t+1}.

value
H[G] 1.9822 (≈ G[S+:S-] = 1.9821)
I[S+ : S- | G] 0
lifted residuals (future, past) 0.0186, 0.0186
any-kernel LP violation 0

So it is a generator state that the lifted search cannot reach; sofic reports 1.991–2.0 there. This PR corrects the docs, docstrings and CHANGELOG:

  • exact and Wyner values are upper bounds over all generators (still valid bounds on C_g: every lifted model is a genuine generator), not minima over all generators;
  • generative_gap can be positive when a non-lifted generator attains the unconstrained value;
  • the functional search is unaffected: for a deterministic channel the coupling is forced, so it is exact over all deterministic generator states.

No behavior change beyond generative_gap. Searching coupled kernels (optimizing p(w_0, w_1 | edge) with fixed marginals) would close the gap and is left for a follow-up.

More starts for the lifted search

Besides S+, S-, (S+, S-), the dit optimum and random channels, the continuation now also starts from:

  • the dit optimum blended with random channels (strengths 0.05 and 0.2, two copies each);
  • time-sharing mixtures of the dit optimum with each ε-machine channel (weights 0.5 and 0.8; disjoint state labels keep the mixture separating).

On the 3-state process 012301 (G = 1.370, ε-machines 1.584) this finds the 1.488 generator in two of three seeds instead of one; on 020213 in one of three. Remaining seed dependence comes mostly from dit's unconstrained optimum varying between runs. Cost: the lifted test files take about 4.6 min instead of 1.6 min locally.

Tests

tests/test_lifted_common_information.py and tests/test_minimal_generative_model.py: 64 passed locally; ruff clean.

Ryan James added 3 commits October 22, 2026 16:25
…tates

GenerativeGap(measure, lower, upper, model) compares G[S+:S-] (or C[S+:S-])
with the same quantity restricted to generator states. For the exact measure
G[S+:S-] <= C_g <= H[G], so gap = upper - lower bounds C_g - G[S+:S-]. Also
available as BidirectionalEpsilonMachine.generative_gap(measure=...).
Lifted generators draw consecutive states independently given the
bidirectional edge; a general generator may couple them. For the interleaved
process Z_t = Z_{t-2} xor Bern(0.1), the product of the chains' minimal
generators (state (U_t, U_{t+1})) is a generator of entropy 1.982, essentially
G[S+:S-], yet violates both lifting conditions, so the lifted search cannot
reach it. State this in the docs, docstrings and CHANGELOG: the exact and Wyner
values are upper bounds over all generators, and generative_gap can be positive
when a non-lifted generator attains the unconstrained value. The functional
search is unaffected (for deterministic channels the coupling is forced).
…it optimum

Besides S+, S-, (S+, S-), the dit optimum and random channels, start the
continuation from the dit optimum blended with random channels (two strengths,
two copies each) and from time-sharing mixtures of it with each eps-machine
channel (weights 0.5 and 0.8; disjoint state labels keep the mixture
separating). On the 3-state process 012301 this finds the 1.488 generator in
two of three seeds instead of one.
@Autoplectic
Autoplectic merged commit 6f25427 into main Oct 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant